With the increasing popularity of digital cameras and digital video cameras, digital images and video have become commonplace. Companies and consumers increasingly depend on computers and computer networks to create, distribute, and play back digital image and video content.
Engineers use a variety of techniques to process digital images and video efficiently while still maintaining quality. To understand these techniques, it helps to understand how image and video information is represented and processed in a computer.
Digital images and digital video consume large amounts of storage and transmission capacity. A typical digital image can include tens or hundreds of thousands of pixels (also called pels). Each pixel represents a tiny element of the picture. In raw form, a computer commonly represents a pixel with 24 bits. A typical video sequence includes 15 or 30 frames per second. Thus, the number of bits per second, or bitrate, of a typical raw digital video sequence can be 5 million bits/second or more.
Many computers and computer networks lack the resources to process raw digital images and video. For this reason, engineers use compression (also called coding or encoding) to reduce the bitrate of digital images and video. Compression can be lossless, in which quality of the images or video does not suffer, but decreases in bitrate are limited by the complexity of the images or video. Or, compression can be lossy, in which quality of the images or video suffers, but decreases in bitrate are more dramatic in subsequent lossless compression. Decompression reverses compression.
In general, image compression techniques are sometimes called intraframe compression techniques. Intraframe compression techniques compress individual images, which are typically called I-frames, key frames, or reference frames when the frames are in a video sequence. Video compression techniques include intraframe compression and interframe compression. Interframe compression techniques compress frames with reference to preceding and/or following frames, and inter-coded frames are typically called predicted frames, P-frames, or B-frames.
For still image compression, well-known intraframe compression standards or products include the various versions of Joint Picture Expert Group [“JPEG”] standards. For compression of I-frames in video, well-known intraframe compression standards or products include the various versions of the Motion Picture Expert Group [“MPEG”] standards, the various H.26x standards from the International Telecommunications Union, and various versions of Windows Media Video [“WMV”] products from Microsoft Corporation. In addition to describing intraframe compression of video frames, the MPEG 4 standard describes encoding and decoding of “sprites,” which are pictures that may be warped in whole or in part for reconstruction at a particular time.
Outside of the area of compression/decompression, various image processing tools allow a user to add effects to still images. When editing an image or series of images (e.g., for a slide show), a user may add effects such as fading, blending, zooming, panning, or rotation. (Fading is essentially blending between an actual image and a blank image.) The effects are presented in images synthesized from the original image or series of images. The entire sequence of original and synthesized images, which is sometimes called a vivid video sequence, may be recorded for distribution and later playback.
Still image compression and video compression successfully reduce the bitrate of still image files for many applications. These forms of compression, however, do not provide adequate performance for vivid video sequences.